Add doc for "Controlling Disruption", document `SafeToEvict` #2924

zmerlynn · 2023-01-25T01:01:12Z

This adds an "Advanced" doc for "Controlling Disruption", documenting the SafeToEvict Alpha.

agones-bot · 2023-01-25T01:31:54Z

Build Failed 😱

Build Id: 8bd3d516-e352-4373-9674-2615ee3a71d3

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-01-25T18:31:07Z

Build Failed 😱

Build Id: 694bc201-3bcc-4c87-8b65-13be3c526568

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-01-25T19:27:09Z

Build Succeeded 👏

Build Id: 1980b9b5-c016-4e68-89c5-7371492cf5e6

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.30.0-a811225-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.30.0-a811225-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.30.0-a811225-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.30.0-a811225-amd64
Linux C++ SDK (build): agonessdk-1.30.0-a811225-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.30.0-a811225-amd64.zip

A preview of the website (the last 30 builds are retained):

https://a811225-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/2924/head:pr_2924 && git checkout pr_2924
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.30.0-a811225-amd64

site/content/en/docs/Advanced/controlling-disruption.md

agones-bot · 2023-01-25T23:19:19Z

Build Succeeded 👏

Build Id: f534a55a-ffdc-4839-9b01-57f3380c65b1

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.30.0-e6647a2-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.30.0-e6647a2-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.30.0-e6647a2-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.30.0-e6647a2-amd64
Linux C++ SDK (build): agonessdk-1.30.0-e6647a2-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.30.0-e6647a2-amd64.zip

A preview of the website (the last 30 builds are retained):

https://e6647a2-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/2924/head:pr_2924 && git checkout pr_2924
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.30.0-e6647a2-amd64

site/content/en/docs/Advanced/scheduling-and-autoscaling.md

roberthbailey · 2023-01-26T23:33:01Z

site/content/en/docs/Advanced/scheduling-and-autoscaling.md

@@ -97,6 +97,12 @@ This affects the Cluster autoscaler, Allocation Scheduling, Pod Scheduling and F

 #### Cluster Autoscaler

+{{< alert title="SafeToEvict Feature Gate" color="info" >}}
+The [Alpha]({{< ref "/docs/Guides/feature-stages.md#alpha" >}}) `SafeToEvict` feature allows


I'm not sure how to best phrase this, but we want to convey that it provides a better way to control disruption than the annotation (which is only for the autoscaler) as it allows the game to establish a contract about it's behavior.

Maybe you cover this more in the longer doc - reviewing this small change first.

I tried to phrase it a little better. WDYT about actually just providing an eviction sample here? i.e. split the samples up with

Using the eviction API

The hard way

or something?

site/content/en/docs/Advanced/controlling-disruption.md

roberthbailey · 2023-01-26T23:44:55Z

site/content/en/docs/Advanced/controlling-disruption.md

+
+* **Ten minutes:** Cluster Autoscaler respects [ten minutes of graceful termination](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#does-ca-respect-gracefultermination-in-scale-down) on scale-down. On some cloud products, you can configure `--max-graceful-termination-sec` to change this, but it is not advised: Cluster Autoscaler is currently only capable of scaling down one node at a time, and larger graceful termination windows slow this down farther (see [autoscaler#5079](https://github.com/kubernetes/autoscaler/issues/5079)). If the ten minute limit does not apply to you, generally you should choose between `safe: Always` (for sessions less than an hour), or see [below](#considerations-for-long-sessions).
+
+* **One hour:** On many cloud products, `PodDisruptionBudget` can only block node upgrade evictions for a certain period of time - on GKE this is 1h. After that, the PDB is ignored, or the node upgrade fails with an error. Controlling `Pod` disruption for longer than one hour requires cluster configuration changes outside of Agones - see [below](#considerations-for-long-sessions).


Do we know what the PBD timeout is on other cloud providers?

On GKE 1h is also the max time given for graceful termination before a pod is forcefully removed during node drain.

From what I can tell gleaning other providers documentations, I think they err on the side of erroring rather than proceeding. I am still trying to research this.

site/content/en/docs/Advanced/controlling-disruption.md

This adds an advanced doc for "Controlling Disruption", documenting the `SafeToEvict` Alpha.

agones-bot · 2023-01-27T22:30:02Z

Build Succeeded 👏

Build Id: f68a50d2-7eea-4438-bb21-207afb2402d0

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.30.0-6c078ce-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.30.0-6c078ce-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.30.0-6c078ce-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.30.0-6c078ce-amd64
Linux C++ SDK (build): agonessdk-1.30.0-6c078ce-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.30.0-6c078ce-amd64.zip

A preview of the website (the last 30 builds are retained):

https://6c078ce-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/2924/head:pr_2924 && git checkout pr_2924
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.30.0-6c078ce-amd64

zmerlynn · 2023-01-27T22:55:56Z

PTAL - restructured this a bit.

agones-bot · 2023-01-27T22:57:58Z

Build Failed 😱

Build Id: 3f5e9d61-f4a8-4261-a3cd-7afd20b1555d

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-01-28T00:02:29Z

Build Succeeded 👏

Build Id: 24261da8-0f7e-4b33-9e39-8a2e528b443e

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.30.0-63eda4f-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.30.0-63eda4f-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.30.0-63eda4f-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.30.0-63eda4f-amd64
Linux C++ SDK (build): agonessdk-1.30.0-63eda4f-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.30.0-63eda4f-amd64.zip

A preview of the website (the last 30 builds are retained):

https://63eda4f-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/2924/head:pr_2924 && git checkout pr_2924
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.30.0-63eda4f-amd64

site/content/en/docs/Advanced/controlling-disruption.md

zmerlynn · 2023-01-31T00:50:43Z

PTAL, updated to include a flowchart diagram and reworked some language.

agones-bot · 2023-01-31T01:00:07Z

Build Failed 😱

Build Id: a82b94ce-524f-4447-ba25-798291806986

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-01-31T01:52:58Z

Build Succeeded 👏

Build Id: 46b58fcf-26b6-4831-9658-421bd9bb0bc1

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.30.0-7947a2c-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.30.0-7947a2c-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.30.0-7947a2c-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.30.0-7947a2c-amd64
Linux C++ SDK (build): agonessdk-1.30.0-7947a2c-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.30.0-7947a2c-amd64.zip

A preview of the website (the last 30 builds are retained):

https://7947a2c-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/2924/head:pr_2924 && git checkout pr_2924
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.30.0-7947a2c-amd64

google-oss-prow · 2023-02-01T01:28:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: roberthbailey, zmerlynn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [roberthbailey]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

google-oss-prow · 2023-02-01T01:28:46Z

New changes are detected. LGTM label has been removed.

agones-bot · 2023-02-01T02:43:29Z

Build Succeeded 👏

Build Id: fc515088-df4f-4ae3-9422-d52cf18958e4

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.30.0-decb99b-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.30.0-decb99b-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.30.0-decb99b-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.30.0-decb99b-amd64
Linux C++ SDK (build): agonessdk-1.30.0-decb99b-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.30.0-decb99b-amd64.zip

A preview of the website (the last 30 builds are retained):

https://decb99b-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/2924/head:pr_2924 && git checkout pr_2924
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.30.0-decb99b-amd64

zmerlynn assigned markmandel and roberthbailey Jan 25, 2023

google-oss-prow bot added the size/L label Jan 25, 2023

google-oss-prow bot requested review from cyriltovena and roberthbailey January 25, 2023 01:01

zmerlynn force-pushed the safetoevict-docs branch from b2cdca8 to 266f35a Compare January 25, 2023 17:20

zmerlynn force-pushed the safetoevict-docs branch from 266f35a to a811225 Compare January 25, 2023 18:45

gongmax reviewed Jan 25, 2023

View reviewed changes

zmerlynn force-pushed the safetoevict-docs branch from a811225 to e6647a2 Compare January 25, 2023 22:35

zmerlynn mentioned this pull request Jan 26, 2023

Graduate SafeToEvict to Beta #2931

Closed

roberthbailey reviewed Jan 26, 2023

View reviewed changes

zmerlynn added 3 commits January 27, 2023 21:51

Add doc for "Controlling Disruption", document SafeToEvict

58a17c7

This adds an advanced doc for "Controlling Disruption", documenting the `SafeToEvict` Alpha.

Address gongmax@ review comments

8bc2a55

Address many of Robby's comments

6c078ce

zmerlynn force-pushed the safetoevict-docs branch from e6647a2 to 6c078ce Compare January 27, 2023 21:51

zmerlynn added 2 commits January 27, 2023 22:47

restructure to a decision tree

bedb9fd

note note: note not a note

63eda4f

roberthbailey reviewed Jan 30, 2023

View reviewed changes

site/content/en/docs/Advanced/controlling-disruption.md Outdated Show resolved Hide resolved

site/content/en/docs/Advanced/controlling-disruption.md Outdated Show resolved Hide resolved

site/content/en/docs/Advanced/controlling-disruption.md Outdated Show resolved Hide resolved

Address Robby's review comments

7947a2c

roberthbailey approved these changes Feb 1, 2023

View reviewed changes

google-oss-prow bot added the lgtm label Feb 1, 2023

Merge branch 'main' into safetoevict-docs

decb99b

google-oss-prow bot added approved and removed lgtm labels Feb 1, 2023

roberthbailey enabled auto-merge (squash) February 1, 2023 01:29

roberthbailey merged commit aa41e4e into googleforgames:main Feb 1, 2023

Kalaiselvi84 added this to the 1.30.0 milestone Feb 28, 2023

Kalaiselvi84 added the kind/feature New features for Agones label Feb 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add doc for "Controlling Disruption", document `SafeToEvict` #2924

Add doc for "Controlling Disruption", document `SafeToEvict` #2924

zmerlynn commented Jan 25, 2023

agones-bot commented Jan 25, 2023

agones-bot commented Jan 25, 2023

agones-bot commented Jan 25, 2023

agones-bot commented Jan 25, 2023

roberthbailey Jan 26, 2023

zmerlynn Jan 27, 2023

roberthbailey Jan 26, 2023

zmerlynn Jan 27, 2023

agones-bot commented Jan 27, 2023

zmerlynn commented Jan 27, 2023

agones-bot commented Jan 27, 2023

agones-bot commented Jan 28, 2023

zmerlynn commented Jan 31, 2023

agones-bot commented Jan 31, 2023

agones-bot commented Jan 31, 2023

google-oss-prow bot commented Feb 1, 2023

google-oss-prow bot commented Feb 1, 2023

agones-bot commented Feb 1, 2023


		* Ten minutes: Cluster Autoscaler respects [ten minutes of graceful termination](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#does-ca-respect-gracefultermination-in-scale-down) on scale-down. On some cloud products, you can configure `--max-graceful-termination-sec` to change this, but it is not advised: Cluster Autoscaler is currently only capable of scaling down one node at a time, and larger graceful termination windows slow this down farther (see [autoscaler#5079](https://github.com/kubernetes/autoscaler/issues/5079)). If the ten minute limit does not apply to you, generally you should choose between `safe: Always` (for sessions less than an hour), or see [below](#considerations-for-long-sessions).

		* One hour: On many cloud products, `PodDisruptionBudget` can only block node upgrade evictions for a certain period of time - on GKE this is 1h. After that, the PDB is ignored, or the node upgrade fails with an error. Controlling `Pod` disruption for longer than one hour requires cluster configuration changes outside of Agones - see [below](#considerations-for-long-sessions).

Add doc for "Controlling Disruption", document SafeToEvict #2924

Add doc for "Controlling Disruption", document SafeToEvict #2924

Conversation

zmerlynn commented Jan 25, 2023

agones-bot commented Jan 25, 2023

agones-bot commented Jan 25, 2023

agones-bot commented Jan 25, 2023

agones-bot commented Jan 25, 2023

roberthbailey Jan 26, 2023

Choose a reason for hiding this comment

zmerlynn Jan 27, 2023

Choose a reason for hiding this comment

Using the eviction API

The hard way

roberthbailey Jan 26, 2023

Choose a reason for hiding this comment

zmerlynn Jan 27, 2023

Choose a reason for hiding this comment

agones-bot commented Jan 27, 2023

zmerlynn commented Jan 27, 2023

agones-bot commented Jan 27, 2023

agones-bot commented Jan 28, 2023

zmerlynn commented Jan 31, 2023

agones-bot commented Jan 31, 2023

agones-bot commented Jan 31, 2023

google-oss-prow bot commented Feb 1, 2023

google-oss-prow bot commented Feb 1, 2023

agones-bot commented Feb 1, 2023

Add doc for "Controlling Disruption", document `SafeToEvict` #2924

Add doc for "Controlling Disruption", document `SafeToEvict` #2924

Using the `eviction` API